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Abstract 

We present a-expansion /3-shrink moves, a 
simple generalization of the widely-used a/3- 
swap and a-expansion algorithms for approx- 
imate energy minimization. We show that in 
a certain sense, these moves dominate both 
a/?-swap and a-expansion moves, but un- 
like previous generalizations the new moves 
require no additional assumptions and are 
still solvable in polynomial-time. We show 
promising experimental results with the new 
moves, which we believe could be used in any 
context where a-expansions are currently em- 
ployed. 



1 Introduction 

We focus on the problem of finding the most probable 
configuration in a pairwise Markov random field over 
discrete variables, a fundamental problem in the study 
of graphical models. This is equivalent to minimizing 
the sum of a set of unary and pairwise energy func- 
tions defined over a set of discrete variables. Problems 
of this type arise in many applications, but in general 
it is NP-hard to solve these problems even in the case 



of binary variables (see Kolmogorov and Zabih 2002 
Theorem 4.2). A classical method for computing an 
approximate solution to this problem is Besag's iter- 



ated conditional mode (ICM) algorithm (Besag 19861, 
but due to the local nature of this method it may be- 
come stuck in a poor local optimum. 

In the special case where the variables are binary and 
the pairwise energy functions satisfy a submodular- 
ity condition, it is possible to solve this problem in 



polynomial-time ( 


Hammer 




1965 


Greig et al. 


1989 


Kolmogorov and Zabih 


2002 


) . This has motivated the 



a/3-swap and a-expansion moves proposed by |Boykov| 



et al.| ( |1998l|1999| ), which we review in |2j These meth- 
ods find an approximate solution to a non-binary prob- 
lem satisfying a submodularity condition over pairs of 
states (or triplets of states in the case of a-expansions) 
by solving a sequence of binary submodular problems. 



The experiments of Szeliski et al. (2008) show that 



these algorithms perform well compared to competing 
approximate minimization methods at minimizing the 
energy functions arising from problems in computer 
vision, such as stereo image matching, creating photo 
montages, and image restoration. Further, this study 
found that a-expansions are often faster than compet- 
ing methods with similar performance for these prob- 
lems (such as variational message-passing algorithms 
like tree-reweighted belief propagation). While orig- 
inally introduced in the context of computer vision, 
these algorithms and their generalizations have also 
proved to be effective in other domains, such as pro- 



tein structure prediction ( Gould et al. 2009 ) 



In a sense that we formally define in Sj3j a/3-swaps 
and a-expansions dominate the classic ICM algorithm. 
However, although a-expansions often lead to better 



experimental results than a/3-swaps ( jSzeliski et al. 



2008), under our definition of dominance neither of 
these more advanced methods dominates the other. In 
this paper we propose a new type of move, a-expansion 
/3-shrink moves (Q- These move are a simple gen- 
eralization of both a/3-swap and a-expansions, that 
dominates them both. Although we delay discussion of 
other generalizations of these moves to the end (|j6j, 
we note that unlike previous generalizations the new 
moves require no additional assumptions beyond those 
needed to apply a-expansions, and the moves can be 
computed in polynomial-time by solving a binary sub- 
modular problem defined on the original graph struc- 
ture. Thus, we can use them in place of a-expansions 
for applications where these moves are currently used. 
Our experiments on standard test problems from the 
field of computer vision (jj5]) show that the new moves 
can lead to improved performance. 
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2 Approximate Energy Minimization 

The problem that we formally address is 

max p(x) oc TT TT <j>ij(x%,Xj), 

1{1,2,...,N}p - lJ - 



xei 



iev 



where V and A are the vertices and arcs of a graph 
(V, .4), while the potentials 4>i(xi) and (j>ij(xi, Xj) map 
assignments of subsets of the discrete vector x to non- 
negative values. If we define the real- valued energy 
functions 



Ei(xi) = -log <f>i(Xi), E i: 



fog (f>ij(xi,Xj), 



then finding the optimal assignment is equivalent to 
the following energy minimization problem: 



min > Ei(xj) 

xe{l,2,...,N}P ^ 



E%j (^i 5 Xj ) , (1) 



In general, solving this optimization problem is NP- 



hard (see Kolmogorov and Zabih 2002 Theorem 4.2), 
and a common approach to finding an approximate 
minimizer is with an iterative descent algorithm. The 
input to each iteration of these algorithms is a par- 
ticular configuration of the variables x, and at each 
iteration an iterative descent method finds a configu- 
ration that minimizes the energy among a set M. (x) of 
possible 'moves', i.e. the next iteration is an element 
of 

argmin^^(y 4 ) + ^ E ij(VhVj)- ( 2 ) 
y£M(x) ieV {i j)eA 

We have two conflicting desiderata on the set of moves 
A4(x k ): we would like this set to be as large as pos- 
sible, but we would like to able to efficiently find the 
optimal move. 

2.1 Iterated Conditional Mode 



In the classic ICM algorithm (Besag 1986), a node j 



is selected and we replace Xj with a value that maxi- 
mizes the conditional probability p(xj \x—j), where X-j 
are the states of all variables except j. In the frame- 
work of energy minimization, this can be viewed as an 
iterative descent method where the elements y of the 
set of possible moves have the form 



7£{1,2,...,7V} if» = j, 
Xj otherwise. 



That is, the current state of node j can be replaced by 
any other possible state, a form of coordinate descent. 
We use A4j(x) to denote the set of all y of this form. 
With this definition of the move space, the iterative 



descent update |2| for the ICM move given x and j 
simplifies to 

argmin Ej(yj\x-j), (3) 

where we will find it convenient to define the condi- 
tional energy of a variable i given a set of variables a 
as 

Ei(xi\x a ) = Ei(xi) 

j\jea,(i,j)eA (4) 

Eij iUi)- 



While exp(—Ei(xi\x-i)) oc p(Xi\x_i), this is a slight 
abuse of the conditioning notation since for other con- 
ditioning sets a it ignores factors that depend on vari- 
ables besides i and those in a. Clearly, we can ef- 
ficiently compute the optimal ICM move by simply 
testing each Ej (yj \x-j). 

2.2 a/3-Swaps 

We say that a pairwise energy function Eij defined 
on binary variables is submodular if it satisfies the in- 
equality 

S y (l,l)+ J B ij -(2,2) <£k(2,l) + 3y(l,2). (5) 

This type of pairwise energy prefers the neighbour- 
ing variables to take the same state. In the special 
case of binary variables where all pairwise energies are 
submodular, the optimal solution to (JlJ can be com- 
puted in polynomial-time as a minimum-cut problem, 
see (Kolmogorov and Zabih 2002 )F] 



In some non-binary problems we have, for all combi- 
nations of states a and /3, that the pairwise energies 
Eij satisfy 

E i j(a,a)+E i j{P,P)<E i j(j3,a)+E ij (a,l3). (6) 

That is, when restricted to any two states a and /?, 
the energy function is submodular. Though we can 
no longer guarantee that we can find the optimal solu- 
tion in polynomial-time given only this restriction, it 



1 Submodularity is normally defined as a property of 
functions on sets, and its use here is because it is 
equivalent to submodularity of a function that takes the 
set of variables labeled 2 and re turns the corresponding 

§7). 



2002 



Eij ( Kolmogorov and Zabih 

There also exist several other notable cases where 
it possible to compute the solution in polynomial-time, 
such as the case where the pairwise energies are con- 



vex ( Ishikawa 2003 1 , or in the case of gene ral energies 



where the graph structure has low tree width dKoller and 
|Friedman| [20091 P3) or is outer-planar (Schraudolph and 



IKamenctsky , 2009). However, these are not our focus and 
a lull discussion or this extensive literature is outside the 
scope of the current work. 



does allow us to take advantage of the ability to effi- 
ciently solve binary submodular problems in order to 
implement a more powerful descent move. 

In particular, given two states a and /3, the set of 
moves M.^p{x) associated with the a/3-swap move in- 
troduced by Boykov et al. ( 1998 ) are of the form: 



a or /? if Xi = a or Xi = f3, 
Xj otherwise. 



That is, the move can simultaneously change any com- 
bination of nodes labeled a to /3, and any combination 
of nodes labeled /3 to a. In this case, the iterative 
descent update is a solution of the problem 



argmm 



Ei(yi\x- a p) 



E 



(i,j)£A\x i ,x j e{a,P} 

where we again make use of our definition of the con- 
ditional energy Q and where we have used X- a p to 
reference the states of the variables not labeled a or f3. 
This is a binary problem over the subgraph induced by 
the nodes labeled a or /?, and under condition ^ this 
update can be computed in polynomial-time because 
all edges in the induced subgraph are submodular. 



for all combinations of states a, 71, and 72. This is a 
stronger condition than Q, which corresponds to the 
special case where 71 =72- If Eij(a, a) = for all a, 
this is the triangle inequality. 



3 Local Dominance of Iterative 
Algorithms 

Some authors analyze the approximation accuracy of 
local optima found by iterative descent methods such 
as a-expansions ( Vekslcr 1999, §4.3.4), but the guar- 



antees on the approximation accuracy tend to be very 
conservative since these global bounds must hold over 
all possible optima. Further, they often do not provide 
guidance in choosing between iterative descent meth- 
ods. For example, a small modification to the analy- 
sis of (see Veksler 1999 §4.3.4) gives us an identical 



global bound for the ICM algorithm. 

Instead of focusing on such a global analysis, we in- 
stead focus on a simple local analysis of the methods. 
Specifically, we are interested in the improvement in 
the objective function that can be obtained in a single 
iteration with different types of moves, given the same 
input to the iteration. From this viewpoint, we say 
that a move set M A {x) dominates a move set M B (x) 
if: 



2.3 a-Expansions 

A closely-related set of moves later proposed 
by Boykov et al. ( 1999 ) are a-expansions. Here, we 



choose a state a and we can use a to replace the cur- 
rent state of any variable. Thus, this set of moves 
M E (x) is of the form 



a if Xi = a, 

a or Xi otherwise. 



We can write the optimal a-expansion as the solution 
to the problem 

argmin 2J Ei(yi\x a ) 

(i.j)£iA\Xi^OL,Xj^OL 

where we use x a to reference variables labeled a. This 
is again a binary problem, this time on the subgraph 
induced by those nodes not labeled a. However, con- 
dition ^ is no longer sufficient to guarantee that the 
edges in the induced subgraph are submodular since 
each pairwise term involves the three states a, Xi, and 
Xj (which may all be different). Nevertheless, it is 
sufficient to ensure that 

J5y(a,a) + £7ij(7i,7 2 ) < £^-(71, a) + £^-(a, 72), (7) 



1. For any energy function E(x) and any input con- 
figuration x, optimizing over M. (x) never gives 
a higher energy than optimizing over Ai B (x): 

Vez, min E(x) < min E(x). (8) 

' y(LM A (x) y€M B (x) 

2. There exists energy functions E(x) and input 
configurations x where optimizing over M A {x) 
gives a strictly lower energy than optimizing over 
M B {x): 

3e x, min E(x) < min E(x). (9) 

' y(LM A (x) yeM B (x) 

This is an intuitive notion of the relative local improve- 
ment that is made by different moves, but it can also 
be used as a measure of the relative 'strengths' of local 
optima found by different types of moves; if the move 
set A4 A (x) dominates the move set A4 B (x), then no 
move in the set Ai B (x) will ever be able to improve 
on a local minimum with respect to move set M. A {x), 
but the optimal move in move set A4 A (x) may be able 
to escape from a local minima with respect to move 
set M B (x). 

To establish dominance relations between various 
search moves, it will be convenient to define versions 



Figure 1: From left to right: Initial labeling, labeling after a/3-swap, labeling after a-expansion, labeling after 
a-expansion /3-shrink. The optimal labeling of the a pixels is outlined by a white triangle, and is achieved from 
the initial labeling by one a-expansion /3-shrink move. 



of the moves that do not depend on any parameters 
except x. Thus, we make the following definitions: 

Ad I (x) = U j Ad I j (x), 
M s {x)=\J a ^M s afj {x), 
Ad E {x) = U ot Ad E (x). 

That is, these expanded move spaces simply search 
over all possible values of the parameter(s). Under 
these definitions, it is straightforward to establish the 
following relationships between the methods: 

Proposition 1. The a/3-swap move set Ad (x) dom- 
inates the ICM move set Ad (x). The a-expansion 
move set Ad E (x) dominates the ICM move set Ad I (x). 

Proof. Assume an arbitrary energy function E and 
configuration x, and let y be the optimal move in 
Ad 1 (x), which must be an element of Adj(x) for some 
j. Then the optimal ICM move is in Ad^g(x) with 
a = Xj and /3 = y,j, which establishes Q. To estab- 
lish ([9]), it is sufficient to construct an energy function 
E along with a configuration x where the optimal so- 
lution involves changing two elements of x. For ex- 
ample, take a two variable problem with x = (1, 1) 
where the optimal solution is (2, 2). Establishing that 
a-expansions dominate ICM can be done similarly. □ 

Proposition 2. The a-expansion move set Ad E (x) 
does not dominate the a/3-swap move setAd s {x). The 
a(3-swap move set Ad s (x) does not dominate the a- 
expansion move set Ad E {x). 

Proof. To establish that a-expansions do not domi- 
nate a/3-swaps, it is sufficient to construct an energy 
function E and configuration x where the optimal so- 
lution is obtained by a single a/3-swap but can not be 
obtained by a single a-expansion. For example, take 
a two- variable problem with x — (1,2) where the op- 
timal solution is (2,1). We can similarly show that 
a/3-swaps do not dominate a-expansions by taking a 
three variable problem with x = (1,2,3) where the 
optimal solution is (1, 1, 1). □ 



Thus, based on the definition of dominance discussed 
in this section, we should prefer both a/3-swaps and a- 
expansions to ICM. However, local dominance does not 
guide us in selecting between these two more advanced 
methods. 

4 a-Expansion /3-Shrink Moves 

Towards the goal of developing a method that dom- 
inates both these methods, consider a generalization 
of both a/3-swaps and a-expansions where the set of 
moves Ad G g (x) are of the form: 



Vi <- 



a or (3 if = a 
a or Xi otherwise 



That is, we allow any node not currently labeled a to 
take the value a, but in addition we allow the nodes 
currently labeled a to take the value /3 (a expands 
everywhere, but is shrunk by /3). We call these a- 



expansion /3-shrink moves, and Figure 2.3 illustrates 
an instance of the move. 

Under the definition of the previous section, we have 
the following result for an analogously defined Ad (x): 

Proposition 3. The a-expansion (3-shrink move set 
Ai G {x) dominates the a(3-swap move setAd s (x). The 
a-expansion (3-shrink move set Ai G {x) dominates the 
a-expansion move set Ad E (x). 

Proof. To establish Q, we note that Ad G (x) includes 
all a/3-swaps and a-expansions as special cases. Fur- 
ther, it also includes moves that are not instances of 
either of these moves (i.e. cases where a expands into 
more than one state, but /3 shrinks into a). Thus, to 
establish ^ we can simply take a three variable prob- 
lem with x = (1,2,3) where the optimal solution is 
(2,1,1). □ 

Note that the argument above can also be used to show 
that the new move set Ad (x) also dominates the move 
set Ad U Ad E , the union of a/3-swap and a-expansion 
moves. 



We can write the optimal a-expansion /3-shrink move 
as a solution to the problem 



argmin ^TEifa) 



Ekjfavj), (10) 



Note that unlike the previous move sets we have dis- 
cussed where the move is computed by solving a prob- 
lem involving conditional energies over an induced sub- 
graph, this move involves solving a problem with the 
original unary energies on the original graph struc- 
ture. In some scenarios, these properties might make 
the generalized move simpler to implement than a/3- 
swaps or a-expansions. Further, if the original graph 
has a special structure, it allows the use of special- 
ized codes for solving binary minimum-cut problems 
with this structure. For example, Dclong and Boykov 
( 2008 ) propose a method for the special case of prob- 



lems with very large grid structures. The simple form 
of the subproblem may also simplify the implementa- 



tion of dynamic graph cuts (Kohli and Torr 20071, 



where the similarity between subproblems when using 
the same value of a (and /?) is used to substantially 
speed up the computation. 

There are many possible moves we could define that 
would dominate one or more of the methods discussed 
in f|2j and we return to this topic in However, an 
appealing property of a-expansion /3-shrink moves is 
the following: 

Proposition 4. // for each edge Eij condition ^ 
holds for all 71 and 72, then all edge energies Eij in 



problem ( 10 ) are submodular. 



Proof. It is sufficient to show this for an arbitrary E^ 
and any possible assignment to Xi and Xj. If Xi =/= a 
and Xj 7^ a, then we require Eij(a, a) + Etj(xi, Xj) < 
Eij(xi,a) + Eij(a,Xj) as with a-expansion moves, 
which is Q with 71 = Xi and 72 = Xj. If Xi = a 
and Xj = a, then we require i?^(a,a) + Eij(J3, /3) < 
Eij(/3,a) + Eij(a,f3) as with a/3-swap moves, which 
is ([7]) with 71 = f3 and 72 = /3. If Xi — a and 
Xj 7^ a, then we require Eij(a,a) + Eij(/3,Xj) < 
Eij(f3,a) + Eij(a,Xj), which is Q with ji = (3 and 
72 = Xj. The remaining possibility is similar. □ 

This implies that an optimal a-expansion /3-shrink 
move can be computed by solving a minimum-cut 



problem, using (for example) the construction of Kol 



mogorov and Zabih (2002). Thus, an optimal a 



expansion /3-shrink move can be computed in poly- 
nomial time under the same condition required to 
use a-expansions. Further, computing an optimal 
a-expansion move and computing an optimal a- 
expansion /3-shrink move have the same worst-case 
time-complexity. This would indicate that the new 



moves could be used in place of a-expansions in any of 
the many applications where these moves are currently 
used. 

4.1 Problems with Many States 

In the computer vision problems that originally mo- 
tivated the moves based on minimum-cuts, the num- 
ber of states N may be non-trivial, since the states 
may represent a discretization of a continuous value. 



For example, Szeliski et al. ( 2008 ) evaluate the perfor- 



mance of these moves on problems with 256 states, 
each state representing an intensity level in an im- 
age. Thus, in addition to their better empirical perfor- 
mance, a-expansions may be preferred over a/3-swaps 
simply because the number of possible values of a is N, 
while the number of (non-exchangeable) combinations 
of a and fi is N(N - l)/2. 

This quadratic scaling in terms of N would also seem 
to be a problem for a-expansion /3-shrink moves. How- 
ever, if we consider any strategy for choosing a value 
of /3 given a, then we still obtain an algorithm that 
dominates a-expansions (assuming we do not always 
choose a = /3, corresponding to the special case of a- 
expansions). Thus, the new moves can be modified to 
have the same scaling with N as a-expansions. That 
is, if ./V is very large we can consider variants where 
/3 is function of a so that we only consider N moves 
instead of N 2 , and these variants would still dominate 
a-expansions. We empirically evaluate three possible 
strategies for selecting /3 given a in fj5] 

4.2 Truncation for Non-Submodular 
Potentials 

In many problems condition |7| is not satisfied. In 
these widely-used approach is to modify the 

potentials to be submodular, in such a way that an op- 
timal move with the modified energy is guaranteed to 



not increase the original energy ( Rother et al. , 2005 1 



In the case of a-expansions, one way to construct such 
a modified energy is by replacing each Eij(xi, Xj) with 

Eij(xi,Xj) = mm{Eij(xi,Xj), 

{fit") 

) + Eij(xi,a) - E l3 (a : a)}. 

Condition ^ holds with this modified energy by con- 
struction. Further, the optimal a-expansion with this 
modified energy does not increase the original energy, 
since the modified energy simply decreases the en- 
ergy of the current assignment (xi ,Xj). As discussed 



in ( jRother et al.| |2005 ), we can alternately increase 
Eij(xi, a) or Eij(a, Xj) to make condition ^ satisfied 
while maintaining the descent property of the moves. 

We can define a modified energy function with sim- 
ilar properties in the case of a-expansion /3-shrink 



moves, though maintaining the descent property re- 
quires a slightly more complicated construction. Al- 
though there are many possible constructions, we de- 
scribe one here. If Xi 7^ a and Xj 7^ a, then as before 
we take: 

E i:j (xi ,Xj) = mm{Eij {x { ,Xj), 

Eij(a,Xj) + Eij(xi,a) - E l3 (a,a)}. 

If Xi = a and Xj = a, then we take: 

Eij(a,a) = mm{Eij(a,a), 

E ij (a,f3)+E ij (l3,a)-E ij (p,p)}. 

If Xi 7^ a and Xj — a, then we take: 

E i:j (a,j3) = max{£ l3 (a,^), 

Eij(a,a) + E i3 (xi,(3) - Eij(xi,a)}. 

Finally, if X4 = a with Xj 7^ a we take: 

Eij{p,a) = m&x{Eij(l3,a), 

Eij(a,a) + Eij(j3,Xj) - Eij(a,Xj)}. 

The other terms in the energy function are unchanged. 
If ([7| is already satisfied, then the modified energy 
under this construction is identical to the original en- 
ergy. This construction maintains the appealing prop- 
erty that any move that does not increase the modified 
energy will not increase the original energy. 

5 Experiments 

To empirically evaluate the performance of the new 
moves, we performed several experiments on the non- 



Table 1: Data sets from 
the experiments. 



Szeliski et al. (2008) used in 



binary data sets examined by Szeliski et al. (2008 1 . 
These data sets are summarized in Table 1, and we 
extracted the terms in the energy functions from the 
code available online 

Our experiments compared the following methods: 



• a/3-Swap: performing a/3-swap moves in the order 
j3 = 1, 2, . . . , JV in an outer loop and a — /3+1, (3+ 
2, . . . , JV in an inner loop. 

• a-Expansion: performing a-expansion moves in 
the order a — 1, 2, . . . , JV. 

• Random /3: performing a-expansion /3-shrink 
moves in the order a = 1, 2, . . . , JV with f3 selected 
randomly among {1,2,..., JV}. 

• j3 = a — 1: performing a-expansion /3-shrink 
moves in the order a = 1, 2, . . . , JV with j3 set 
to max{l, a — 1}. 



Name 


Task 


Nodes 


Edges 


States 


Family 


Montage 


425632 


849946 


5 


Pano 


Montage 


514080 


1026609 


7 


Tsukuba 


Stereo 


110592 


220512 


16 


Venus 


Stereo 


166222 


331627 


20 


Teddy 


Stereo 


168750 


336675 


60 


Penguin 


Restoration 


21838 


43375 


256 


House 


Restoration 


65536 


130560 


256 



"http : //vision . middlebury . edu/MRF/ 



• j3 — a + 1: performing a-expansion /3-shrink 
moves in the order a = 1,2, ... ,N with j3 set 
to min{JV, a + 1}. 

• All j3: performing a-expansion /3-shrink moves in 
the order j3 — 1,2, ... ,N in an outer loop and 
a = 1, 2, . . . , JV in an inner loop. 

Note that the iterations of the first and last method 
are much more expensive for large JV because they 
consider 0(N 2 ) combinations of a and j3, while the 
remaining methods only consider JV moves. 

In our first experiment, we initialized all variables to 
the first state and ran each method until the energy did 
not change between iterations. We used the truncation 
described in { 4.2 for problems that did not satisfy ^ 
for all triplets of states. In Table 2, we show the energy 
of the local minima obtained divided by the energy of 
the local minimum with respect to a-expansion moves. 
In this table, a value of 1 indicates that the energy 
was identical to the energy obtained by a-expansion 
moves, and we use 1.0000 if the energy is close but 
not identical. In this experiment, the new moves with 
all (3 obtained the lowest energy on 6 of the 7 data 
sets, and strictly so in 4 of these cases (the exceptions 
were Teddy where using a random (3 lead to a lower 
score, and the montage data sets where other meth- 
ods reached the same energy). Among the remaining 
methods, the more computationally efficient strategy 
of simply setting f3 = a + 1 obtained the lowest energy 
on 5 of the 7 data sets. 

In our second experiment we used the local minimum 
with respect to a-expansion moves as the initialization, 
and tested whether the methods that only consider JV 
moves could escape from this local minimum. Table 3 
shows the energies of the local optima obtained with 
this initialization. Here, we see that even with these 
simple choices of f3 that the new moves are able to es- 
cape the local minimum with respect to a-expansions 
for 5 out of the 7 data sets. In this experiment choos- 
ing f3 = a + 1 gave the lowest energy on all data sets, 
and gave a strictly lower energy on 4 of the 7. Select- 
ing j3 = a + 1 is somewhat intuitive, the moves can 



Table 2: Energy of local minima with respect to iterative descent methods beginning from all variables set to 
state 1, divided by the energy of the local minima with respect to a-expansion moves. 



Name 


a/3- Swap 


a-Expansion 


Random j3 


/3 = a- 1 


(3 = a + l 


All /? 


Family 


1.0203 


1 


0.9998 


1 


0.9998 


0.9998 


Pano 


1.3182 


1 


1.0006 


1 


1 


1 


Tsukuba 


1.0315 


1 


1.0012 


1 


1.0000 


1.0000 


Venus 


1.8561 


1 


1.0015 


0.9992 


0.9979 


0.9968 


Teddy 


1.0037 


1 


0.9998 


1 


1.0007 


0.9999 


Penguin 


1.1283 


1 


1.0037 


0.9936 


0.9793 


0.9758 


House 


0.7065 


1 


0.7841 


0.9973 


0.7038 


0.7032 



Table 3: Relative energy of local minima for different 
choices of f3 when initializing with a local minima with 
respect to a-expansions. 



computed in polynomial-time without further assump- 



Name 


Random j3 


/3 = a- 1 


P = a + 1 


Family 


0.9998 


1 


0.9998 


Pano 


1 


1 


1 


Tsukuba 


1 


1 


1 


Venus 


1.0000 


0.9992 


0.9979 


Teddy 


1 


1 


0.9999 


Penguin 


0.9998 


0.9902 


0.9775 


House 


0.8050 


0.9971 


0.7038 



be viewed as 'prematurely' expanding the next value 
of a into the region occupied by the current value of 
a. Although an improved configuration was found on 
at least one data set for each of the three tasks, the 
largest improvements were seen in the image restora- 
tion/inpainting tasks. This is likely due to the 256 
possible labels, which is much larger than the other 
tasks. In Figure [5j we show the restoration of the 
house data where the largest improvement was ob- 
served. In this figure we see a noticeable difference 
between the local minima with respect to the different 
moves. This visual difference is also reflected quanti- 
tatively; the new moves lead to a reconstruction er- 
ror (^i-norm distance to the original image) that is 
approximately two-thirds that of the local minimum 
with respect to a-expansions (reduced from 1303537 
to 911207). 

6 Discussion 

There have been a variety of other generalizations of 
a/3-swaps and a-expansions proposed in the litera- 
ture, and an exhaustive list of this literature would 
be outside our scope. However, among the gener- 
alizations we are aware of these other moves either 
(i) require additional assumptions, or (ii) can not 
be solved in polynomial-time. For example, range- 
swap and range-expansion moves generalize a/3-swaps 
and a-expansions (respectively), but these can not be 



tions (Veksler 2007 Gould et al. 2009 Kumar et al. 



2011). Kolmogorov and Rother (20071 consider imple- 



menting minimum-cut methods wihout restrictions on 
the energy functions, but in general their algorithm 
can not compute the optimal move. |Lem pitsky et al.| 
(|2010p consider a general class of moves called fusion 



moves. These moves consider two full configurations 
of the variables x and z, and the set of moves are of 
the form 

Hi <- | a;, or z,. 

That is, we can replace an arbitrary number of ele- 
ments of x by the corresponding elements from some 
alternative configuration z. The a-expansion /3-shrink 
moves proposed here are a special case of this type of 
move, where Zj is set to j3 if x$ = a and Z; is set to 
a otherwise. However, while Proposition 4 shows that 
this special case is solvable in polynomial-time under 
condition ([7]) , |Lempitsky et"aL (2010) do not give re- 
strictions that would allow an optimal general fusion 
move to be solved in polynomial time. Indeed, because 
of the generality of fusion moves, any such restriction 
would need to be much more restrictive than Q. 

Although in this paper we have focused on the case of 
minimizing unary and pairwise energies, it is possible 
to apply a/3-swaps and a-expansions to certain classes 
of higher-order energies (Kohli et al. 2007). We ex- 



pect that the new moves discussed in this paper can 
also be extended to these scenarios, and indeed we be- 
lieve that the new moves could be used in place of 
a-expansions in any of the many applications where 
these moves are currently used. Finally, we note that 
the implementation of the new moves will be made 
available online. 
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